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Executive Summary 


This white paper is intended as an additional tool, specifically an infrastructure sizing and 
testing guide, to assist organizations who are looking to successfully implement 
Microsoft® Business Solutions CRM version 1.2 in large scale deployments. It is designed 
to provide information that is complementary to that already included in the Microsoft 
Business Solutions CRM Implementation Guide. 


This paper is not to be confused with a scalability benchmark- its goal is not to set a 
threshold for the maximum number of users supported under the test hardware 
configuration. The specific scenarios tested demonstrate the ability of Microsoft CRM to 
scale to 1,000 system users while preserving excess system capacity, but the results and 
the testing methodology are useful for deployments both smaller and larger than 1,000 
users. 


Testing was conducted for Microsoft CRM Sales Standard version 1.2 running on 
Microsoft Windows Server™ 2003 and Microsoft SQL Server™ 2000. This performance 
test did not include Microsoft CRM Sales for Outlook (the Outlook client) offline scenarios 
or the replication of disparate Microsoft SQL Server 2000 Desktop Engine (MSDE 2000) 
databases; however, use cases applied in this test can be leveraged in both the Microsoft 
CRM Web client and the Outlook client running in online mode. A separate performance 
test has been commissioned to include replication of offline Outlook clients. 


This performance test was performed at the Microsoft campus in Redmond, WA by a 
team of Microsoft employees. 


Deployment Considerations 


While this performance test represents a fixed workload in a static test environment, the 
methodology created for this performance test may be used to examine varying 
deployment workloads for Microsoft CRM. As with all successful Microsoft CRM 
deployments, customers and partners must understand their unique usage requirements 
and incorporate this usage into their performance tests. 


Performance Test Design and Measures 


The Microsoft test team designed this performance test focusing on realistic 
implementations of Microsoft CRM. Test scenarios and workloads were derived from a 
sampling of Microsoft CRM customer deployments, partner interviews, and standard 
sales force automation use-cases provided by industry analysts. Hardware configuration 
was selected based on the type of infrastructure you might expect to find within an IT 
organization of 1,000 sales force automation users. 


Though the constructed workflow and hardware configuration attempts to model a 
baseline, real-world Microsoft CRM implementation, it does not attempt to model all 
possible hardware configurations and system usage scenarios of actual system 
deployments. 


The performance test was constructed to gauge the following: 


e Measure the availability (percent utilized) and reliability (percent accurate) of 
Microsoft CRM throughout the eight-hour test interval. 


e Record Microsoft CRM version 1.2 application response under medium-heavy 
workload scenarios. 


e Monitor Microsoft CRM database transactions and CPU utilization on SQL Server 2000. 


e Measure Microsoft CRM Web server traffic and CPU utilization on Windows Server 
2003. 


Summary of Results 


Microsoft test professionals tested Microsoft CRM version 1.2 with 1,000 virtual users 
performing a medium-heavy sales force automation workload over an 8-hour period. The 
following data was collected using System Monitor and Microsoft Application Center Test 
(ACT): 


e The Microsoft CRM Web application server had an average CPU utilization of 9.66 
percent leaving roughly 90 percent of CPU available to manage additional processing 
overhead. 


e Average processor utilization of the Microsoft CRM database server was 60.71 
percent, leaving approximately 40 percent of CPU available to manage additional 
overhead. 


e Microsoft CRM was 99.998 percent accurate within the 8-hour test period (4 errors? 
out of 222,899 test transactions). 


Test results are a fixed representation of the hardware environment, workload, and 
network traffic experienced during the actual test. Changes to these variables may 
generate different results within subsequent performance tests. 


Hardware Configuration Summary 


As enumerated within the test methodology, the server infrastructure and hardware 
configuration was congruent with the infrastructure and configuration that is likely to be 
found in a customer deployment for 1,000 system users. The following figure shows the 
hardware configuration that was used in the performance test. 


1 Microsoft test professionals measured response code 500, “The server encountered an unexpected condition 
that prevented it from fulfilling the request” when calculating system errors. 


Windows Server 2003 Domain Controller 


Manufacturer: HP 

Model: Compag Proliant DL360 

Operating System: Microsoft Windows Serer 2003 
Processor: 2-1 GHZ Pentium Ill 

Memory: 512 MB 

Software: Active Directory 


Microsoft CRM Web Application Server 


Manufacturer: HP 

Model: Compaq Proliant DL380 G3 

Operating System: Microsoft Windows Serer 2003 

Processor: 2-2.4 GHZ Intel Xeon -Hyperthreading Enabled 

Memory: 512 MB of RAM 

Network 1 GB Ethernet Adapter 

Software: Microsoft Business Solutions CRM v1.2 US English, IIS6.0, .NET 
Framework v 1.1 


CRMWEB01 


Microsoft CRM Database Server 


Manufacturer: Hewlitt Packard 
Model: Compaq Proliant DL580 G2 
Operating System: Microsoft Windows Sener 2003 
Processor: 4 — 1.6 GHZ Intel Xeon Processor with Hyperthreading Enabled 
Memory: 2 GB of RAM 
CRMSOLO01 Network 1 10/100 Ethernet Adapter 
Software: Microsoft SQL Server 2000 SP3 


Microsoft Application Center Test Client (x2) 


Manufacturer: HP 

Model: Compag Proliant DL360 

Operating System: Microsoft Windows 2000 

Processor: 2-1 GHZ Pentium III 
SS Memory: 512 MB 


ACTClient ACTClient Software: Microsoft Application Center Test (ACT), PerfMon 


Performance Test Details 


Two objectives were identified for undertaking this performance test. The first objective 
was to provide system utilization and performance metrics for a Microsoft CRM 
deployment of 1,000 sales force automation users performing a medium-heavy 
transaction mix. The second objective was to provide infrastructure guidance for 
customers who want to successfully deploy Microsoft CRM under similar usage 
requirements. 


The remainder of this paper describes the details and results of this performance test. 


Test Framework 


Rather than develop a specific tool for measuring the performance of Microsoft CRM 
under the test workload, the team used Microsoft Application Center Test (ACT) to collect 
performance data. Microsoft ACT is a utility that can be used to stress test Web servers 
and measure application performance; it simulates system usage by opening multiple 
connections to the Web server and rapidly sending HTTP requests to be handled by the 
Microsoft CRM Web server. 


Microsoft ACT is available for installation as part of the Enterprise Features of Microsoft 
Visual Studio® .NET. For more information about Microsoft ACT, see the article 
“Microsoft Application Center Test 1.0, Visual Studio .NET Edition” on the Microsoft 
Developer Network (MSDN) Web site 
(msdn.microsoft.com/library/default.asp?url=/library/en-us/act/htm/actml_main.asp). 


User Simulation 


Microsoft ACT utilized two physical workstations- each simulating the requests of 500 
Microsoft CRM system users. Workload and server requests were randomized to model 
system usage so that operations of the same type were not performed simultaneously. In 
addition, data access and retrievals were performed in an indiscriminate manner to 
prevent artificial caching of Web and database resources. Operations were performed at 
the actual speed that a system user would perform work activities, making this 
performance test comparable to the system usage of 1,000 physical client computers 
over a simulated 8-hour workday. 


Test Hardware Configuration 


The following tables list the technical environment that was used within the Microsoft 
CRM version 1.2 performance test. 


Virtual client simulators (ACTClientl and ACTClient2) 


Manufacturer Hewlett Packard 

Model Proliant DL360 

Operating System Microsoft Windows® 2000 Advanced 
Server 

Processor (2) 1-GHz Intel Pentium III 

RAM 512 MB 

Software Microsoft Application Center Test (ACT) 


System Monitor 


Database server (CRMSQLO1) 


Manufacturer Hewlett Packard 
Model DL580G2 
Operating System Microsoft Windows Server 2003 


Processor (4) 1.6-GHz Intel Xeon with hyper-threading 


enabled 

RAM 2 GB 

Software Microsoft SQL Server 2000 SP3a 

Disk Configuration? Disk 0: 16.95 GB, O/S, RAID 1 

(4 logical partitions) Disk 1: 16.95 GB, SQL Server log files, 
RAID 0+1 


Disk 2: 25.42 GB, Database Backup, RAID 5 


Disk 3: 33.91 GB, SQL Server database, 
RAID 0+1 


Microsoft CRM Web/platform server (CRMWEBO1) 


Manufacturer Hewlett Packard 

Model Proliant 380 G2 

Operating System Microsoft Windows Server 2003 

Processor (2) 2.4-GHz Intel Xeon with hyper-threading 
enabled 

RAM 512 MB 

Software Internet Information Services (IIS) 6.0 
Microsoft CRM Server 

Network Adapter 1 Gigabit Ethernet Adapter 


Domain controller (CRMDCO1) 


Manufacturer Hewlett Packard 

Model Proliant DL360 

Operating System Microsoft Windows® 2000 Advanced 
Server 

Processor (2) 1-GHz Intel Pentium III 

RAM 512 MB 

Software Active Directory ® 

Network 

Network Cisco Catalyst 2980G Gigabit Ethernet 


? The Microsoft SQL Sever database was configured according to Microsoft hardware engineering standards. 


Microsoft CRM Test Database 


Initial data loading of the Microsoft CRM test database was extrapolated by analyzing the 
sample sales force automation workload and pre-populating the Microsoft CRM system 
with the equivalent of 3 months of customer data. The following table lists the resulting 
database population by Microsoft CRM object. 


Microsoft CRM object Initial data volume 


Task activity 125,876 
Fax activity 122,960 
Phone activity 125,198 
E-mail activity 124,498 
Letter activity 121,377 
Appointment activity 119,758 
Accounts 40,698 

Leads 162,769 
Contacts 81,481 

Opportunities 26,710 


Microsoft CRM Environment Setup 


Test setup centered on the premise of an actual customer implementation of Microsoft 
CRM Suite Professional. Microsoft test professionals constructed an Active Directory 
topography similar to that found in a customer deployment and assigned appropriate 
Microsoft CRM roles and privileges to virtual users that were consistent with test use- 
cases. The following table depicts the Microsoft CRM setup variables used in the 
performance test. 


Microsoft CRM setup variables 


Value 


Organizational unit 
Number of licenses 


License type 


Security roles assigned (user count) 


Adventure Works Cycle 
1,000 


Microsoft Business Solutions CRM 
Sales Standard 


System Administrator (5) 
Salesperson (973) 

Sales Manager (20) 

Vice President of Sales (1) 


CEO-Business Manager (1) 


Performance Test Use-Cases 


Through customer and industry analyst interviews, the test team established a sales 
force automation workflow that reflects medium-heavy system usage. Microsoft test 
professionals did not create artificial fluctuations in the test workload that would 
represent beginning of shift/end of shift usage or offline synchronization of data to 
Microsoft CRM Sales for Outlook (the Outlook client). Rather, use-cases were distributed 
evenly between the 1,000 virtual users to eliminate unnecessary bias and caching within 
the Microsoft CRM Web application. 


Microsoft CRM Accounts 


Microsoft CRM accounts represent businesses with which an organization cultivates a 
relationship. They may characterize existing customers, potential customers, and 
business partners such as vendors and suppliers. Within the performance test workflow, 
Microsoft CRM accounts were defined in the context of a sale-customer. 


Count per 
user per 8- Total test 
Description Operation hour shift transactions 


Creation of a Microsoft 
CRM account record. Create 1 1,000 


Account retrieval. Retrieve 12 12,000 


Retrieval of an account 
record by object ID and 
a direct modification of 
the object properties. RetrieveUpdate 8 8,000 


Retrieval of an account 

record through the 

object’s list view and 

display of the object’s 

property form. SearchRetrieve 16 16,000 


Retrieval of an account 

record through the 

object’s list view and 

display of the object 

property form. State 

code (Status field) is 

updated to “Inactive.” SearchRetrieveDeactivate 0.3 300 


Retrieval of an account 

record through the 

object’s list view and 

display of the object’s 

property form. 

Modifications made and 

saved. SearchRetrieveUpdate 12 12,000 


Microsoft CRM Contacts 


Microsoft CRM contacts are the individual parties, or personas, that define the business 
relationship. Within the context of the performance test, Microsoft CRM contacts were 
used to manage relationships with business acquaintances. 


Count per user Total 
per 8-hour iterations 
Description Operation shift 


Creation of a Microsoft 
CRM contact record. Create 2 2,000 


Contact retrieval. Retrieve 12 12,000 


Retrieval of contact 

record by object ID and 

a direct modification of 

the object properties. RetrieveUpdate 8 8,000 


Retrieval of a contact 

record through the 

object’s list view and 

display of the object’s 

property form. SearchRetrieve 16 16,000 


Retrieval of a contact 

record through the 

object’s list view and 

display of the object 

property form. State 

code (Status field) is 

updated to “Inactive.” SearchRetrieveDeactivate 0.5 500 


Retrieval of a contact 

record through the 

object’s list view and 

display of the object’s 

property form. 

Modifications made and 

saved. SearchRetrieveUpdate 12 12,000 


Microsoft CRM Leads 


Microsoft CRM leads represent unqualified, potential sale-customers that have been 
identified by an organization’s sales force. The performance test workflow defined the 
use of Microsoft CRM leads as potential customers while test workflow used leads in the 
manner described in the following table. 


Count per Total 
user per 8- iterations 
Description Operation hour shift 


Creation of a Create 4 4,000 


Microsoft CRM lead 
record. 


Lead retrieval. Retrieve 12 12,000 


Retrieval of lead 

record by object ID 

and a direct 

modification of the 

object properties. RetrieveUpdate 8 8,000 


Retrieval of a lead 

record through the 

object’s list view 

and display of the 

object’s property 

form. SearchRetrieve 16 16,000 


Retrieval of a lead 

record through the 

object’s list view 

and display of the 

object property 

form. State code 

(Status field) is 

updated to 

“Inactive.” SearchRetrieveDeactivate 0.5 500 


Retrieval of a lead 

record through the 

object’s list view 

and display of the 

object’s property 

form. Modifications 

made and saved. SearchRetrieveUpdate 12 12,000 


Microsoft CRM Opportunities 


Microsoft CRM opportunities track the likelihood of making a sale to a potential sales 
customer. The performance test workflow used opportunities in the manner described in 
the following table. 


Count per 
user per 8- 
Description Operation hour shift Total iterations 
Creation of a 
Microsoft CRM 
opportunity 
record. Create 0.5 500 


Opportunity 
retrieval. Retrieve 6 6,000 


Retrieval of an 
opportunity record 
by object ID anda 
direct modification 
of the object 


properties. RetrieveUpdate 


Retrieval of an 
opportunity record 
through the 
object’s list view 
and display of the 
object’s property 


form. SearchRetrieve 


Retrieval of an 
opportunity record 
through the 
object’s list view 
and display of the 
object property 
form. State code 
(Status field) is 
updated to 
“Inactive.” 


Retrieval of an 
opportunity record 
through the 
object’s list view 
and display of the 
object’s property 
form. 
Modifications 


made and saved. SearchRetrieveUpdate 


Microsoft CRM Activities 


SearchRetrieveDeactivate 


4,000 


8,000 


0.2 200 


The activity management functionality in Microsoft CRM enables salespeople to track 
their personal work as it relates to customer interactions. The following table describes 
the activity management transaction mix that was incorporated into the performance 


test workflow. 


Count per 
user per 8- 
Description Operation hour shift Total iterations 
Appointment 
activity creation. Create 2 2,000 
E-mail activity 
creation. Create 12 12,000 


Fax activity 
creation. Create 1 1,000 


Letter activity 
creation. Create 1 1,000 


Phone activity 
creation. Create 8 8,000 


Task activity 
creation. Create 8 8,000 


Workload Summary 


With 1,000 Microsoft CRM users performing the previously mentioned medium-heavy 
sales force automation workload, the following table provides a detailed estimate of daily 
and hourly operations that were loaded into the performance test. 


Count per 
Measure user Total iterations 
8-hour test total 209 209,000 
Operations per hour 26.13 26,130 


Microsoft CRM Customization 


No deployment-specific customizations were made to the Microsoft CRM installation. 
There were no Microsoft CRM XML schema changes, application customizations, workflow 
rules, or synchronizations used in the performance test. The test team used a clean 
install of Microsoft CRM and accepted all of the default installation parameters?. 


Deletion Service 


The Microsoft CRM Deletion Service was disabled for this test because customer and 
analyst interviews found that few installations enabled the physical deletion of customer 
information. On average, interviewed customers preferred logical deletions, or 
deactivations, of customer information in order to preserve the historical data. 


Performance Tuning 


No performance tuning configurations were made to any of the servers within the 
performance test. All server applications were installed with default settings. 


Microsoft CRM Performance Test Results 


As previously stated, this performance test was conducted to demonstrate the ability of 
Microsoft CRM to scale to 1,000 system users while preserving excess system capacity. 
This performance test was not engineered to formulate a system benchmark of the 
maximum number of users able to use Microsoft CRM within a customer deployment. 
Over an 8-hour performance tests, findings concluded that: 


3 A customized Microsoft CRM system may produce different results under the test workload. 


e The Microsoft CRM Web application server (CRMWEBO1) held an average CPU 


utilization of 9.66 percent. 


e The Microsoft CRM database server (CRMSQLO1) maintained an average utilization of 


60.71 percent. 


e Microsoft CRM was 99.998 percent accurate within the 8-hour test period (4 errors‘ 


out of 222,899 test transactions). 


e Average perceived response time across all operations for system end-users was 


1.56 seconds?. 


Application Center Test Summaries 


Microsoft ACT was used on two physical workstations and collected the following data 
from the Microsoft CRM performance test. Summarized information includes the total 
number of requests, cumulative error counts, and response codes generated by the 


Microsoft CRM Web application. 


The following tables summarize the test properties and results for the ACTClient1 test. 


ACTClientl Summary 


Test name 
Test run name 


Test started 


Microsoft CRM Performance Test 
1000 Users-8 Hours 
12/4/2003 7:23:15 PM 


Test duration 00:08:00:00 
Test iterations 111,296 
Test notes None 
Properties 

Test type Dynamic 
Simultaneous browser connections 500 

Warm up time in seconds (sec) 900 

Test duration 00:08:00:00 
Test iterations 111,296 
Detailed test results generated Yes 


4 Microsoft test professionals measured response code 500, “The server encountered an unexpected condition 
that prevented it from fulfilling the request” when calculating system errors. 


° Perceived application response time is defined as Microsoft ACT TTLB plus Internet Explorer rendering time. 


Summary 


Total number of requests 293,176 
Total number of connections 293,349 
Average requests per second 10.18 
Average time to first byte in milliseconds 1,152.54 
(msec) 

Average time to last byte (msec) 1,555.74 
Average time to last byte per iteration 4,098.13 


(msec) 


Number of unique requests made in test 23 


Number of unique response codes 2 


Error counts 


HTTP (0) 
DNS 0 
Socket 346 


Additional network statistics 


Average bandwidth (bytes/sec) 216,387.32 
Number of bytes sent (bytes) 853,998,868 
Number of bytes received (bytes) 5,377,955,960 
Average rate of sent bytes (bytes/sec) 29,652.74 


Average rate of received bytes (bytes/sec) 186,734.58 


Number of connection errors 0 
Number of send errors 173 
Number of receive errors 173 
Number of timeout errors 0 


Response codes 


Response Code: 200 - The request completed successfully. 
Count 293,186 
Percent (%) 99.99 


Response Code: 1xx - Custom Codes 


Count 19 


Percent (%) 0.01 


The following tables summarize the test properties and results for the ACTClient2 test. 


ACTClient2 Summary 


Microsoft CRM Performance Test 
1000 Users-8 Hours 
12/4/2003 7:23:04 PM 


Test name 
Test run name 


Test started 


Test duration 00:08:00:00 
Test iterations 111,603 
Test notes None 
Properties 

Test type Dynamic 
Simultaneous browser connections 500 

Warm up time (sec) 900 

Test duration 00:08:00:00 
Test iterations 111,603 
Detailed test results generated Yes 
Summary 

Total number of requests 293,193 
Total number of connections 293,357 
Average requests per second 10.18 
Average time to first byte (msec) 1,147.32 


Average time to last byte (msec) 1,548.35 


Average time to last byte per iteration 


4,067.67 
(msec) 


Number of unique requests made in test 23 


Number of unique response codes 3 


Error counts 


HTTP 4 
DNS 0 
Socket 328 


Additional network statistics 


Average bandwidth (bytes/sec) 216,444.88 
Number of bytes sent (bytes) 854,092,325 
Number of bytes received (bytes) 5,379,520,363 
Average rate of sent bytes (bytes/sec) 29,655.98 


Average rate of received bytes (bytes/sec) 186,788.90 


Number of connection errors 0 
Number of send errors 164 
Number of receive errors 164 
Number of timeout errors 0 


Response codes 


Response Code: 500 - The server 
encountered an unexpected condition that 
prevented it from fulfilling the request. 


Count 4 
Percent (%) 0.00 


Response Code: 200 - The request 
completed successfully. 


Count 293,194 
Percent (%) 99.99 


Response Code: 1xx - Custom Codes 
Count 25 
Percent (%) 0.01 


Server Processor Utilization 


Microsoft Windows System Monitor was used to capture processor utilization for the 
Microsoft CRM Web and database servers. The following chart shows server processor 
utilization while all 1,000 users performed the test use-cases over the 8-hour test period. 


ai 
& 
® 
$ 

] 
D 
© 
e 
a 
y 


Lat 0.000 Average 60.706 Mirau 36 205 Macerrem 77.858 Duraton 74805 


——=— | 000 %6 Precespor Time Total Proce. Ucrmperfweba0] 


meeer 1000 H Processor Teme Toul = Proce. VESO 


Processor utilization for the 8-hour performance test produced the following results. 
Server Counter Value 


CRMWEBO1 % Processor Time (Avg.) 9.657 


% Processor Time (Min) 6.912 


% Processor Time (Max) 20.392° 
CRMSQLO1 % Processor Time (Avg.) 60.706 
% Processor Time (Min) 36.205 
% Processor Time (Max) 77.858 


Web Requests and Transactions Per Second 


System Monitor also collected the Web requests and database transactions per second 
that occurred on the Microsoft CRM Web and database servers. The following chart 
shows the resulting measurements of each that resulted from 1,000 users performing 
test use-cases over the 8-hour period. 
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The number of requests/transactions per second from the 8-hour performance test 
produced the following results. 


ê This outlier was caused by the indexing service recalculating indexes for Microsoft CRM knowledge base 
articles. 


Server Counter Value 


CRMWEBO1 Requests/sec (Avg.) 20.367 
Requests/sec (Min) 16.367 
Requests/sec (Max) 23.837 
CRMSQLO1 Transactions/sec (Avg.) 13.679 
Transactions/sec (Min) 10.327 
Transactions/sec (Max) 18.317 
Conclusion 


This performance test demonstrated the ability of Microsoft CRM to scale to 1,000 
system users while preserving excess system capacity. Processor utilization of the 
Microsoft CRM SQL Server database server averaged 60.71 percent and never exceeded 
77.86 percent, while processor utilization on the Microsoft CRM Web application server 
averaged 9.66 percent and never exceeded 20.39 percent’. Test cases assumed a 
medium-heavy sales force automation transaction mix while sustaining average 
perceived response times of less than 1.56 seconds’®. 


7 A customized Microsoft CRM system may produce different results under the test workload. 
8 Perceived application response time is defined as Microsoft ACT TTLB plus Internet Explorer rendering time. 


